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(57) Abstract 



A mechanism for determining parallax between first and second digital images employs an iterative image pattern search 
and correlation process, which is able to rapidly determine the parallax for each pixel that is common to the images captured by a 
stereo image pair. By the initial use of a dispersed sub-array of reference point pixels distributed throughout a first image array, a 
similarly dispersed sub-array of tie points, to be used as starting points for subsequent, more densely populated search and corre- 
lation steps, can be rapidly obtained for the second image array. The derived parallax values may then be subjected to an interpo- 
lation function to sub-pixel accuracies. Once the parallax between the first and second image arrays has been derived, elevations 
can be computed using a conventional pcra!!^ dependent "chin:™. 



FOR THE PURPOSES OF INFORMATION ONLY 



applicaJSSfuS'ih^pff Sta,CS ^ l ° PCT °" fr0m ***** ° f P am P h,ete Publishing international 



AT Austria 

AU Australia 

BB Barbados 

BE Belgium 

BP Burkina Faso 

BG Bulgaria 

BJ Benin 

BR Brazil 

CA (Canada 

CF (Antral African Republic 

CC Congo 

CH Switzerland 

CI Coted'lvoire 

CM Cameroon 

CS Czechoslovakia 

DE* Germany 

OK Denmark 



ES Spain 

Fl Finland 

FR France 

CA Gabon 

CB United Kingdom 

CN Guinea 

CR Greece 

HU Hungary 

IT Italy 

JP Japan 

KP Democratic People's Republic 

of Korea 

KR Republic of Korea 

LI Liechtenstein 

LK Sri Lanka 

LU Luxembourg 

MC Monaco 



MC 


Madagascar 


ML 


Mali 


MN 


Mongolia 


MR 


Mauritania 


MW 


Malawi 


NL 


Netherlands 


NO 


Norway 


PL 


Poland 


RO 


Romania 


SD 


Sudan 


se 


Sweden 


SN 


Senegal 


SU+ 


Soviet Union 


TD 


Chad 


TC 


Togo 


US 


United Stales of America 



+ Any designation of M SU" has effect in the Russian Federation. It is not yet known whether 
any such designation has effect in other States of the former Soviet Union. 



WO 92/06444 W W PCT/US9 1/06908 



10 



MECHANISM FOR DETERMINING PARALLAX 

BETWEEN DIGITAL IMAGES 

FIELD OF TWff TWFMTTnrc 

The present invention relates in general to 
stereo image processing and is particularly directed to 
an iterative search and correlation mechanism for 
determining the parallax between respective digital 
images of a contoured surface. 
BACKGROTTNT) OF TWF ™YFINTTTT 

Stereo images of a contoured surface (e.g. 
terrain) are often employed in elevation mapping and 
three dimensional visualization applications to derive 
elevation data from which a perspective view of the 
surface from an arbitrary vantage point can be 
15 recreated, in order to determine elevation information 
from a two dimensional image, the parallax between 
common locations of the stereo image pair is measured. 
As an example of how parallax may be used to determine 
the elevation of objects within a scene, consider the 
20 diagrammatical illustrations, in Figure 1, of 

respective images 11 and 12 containing a pyramid 14, as 
viewed from respectively different look angles in the 
horizontal (or x) direction. The pyramid has four 
corners A, B, c and D that lie on generally flat 
25 surface 16 and an apex E that H R »q 

— ww**»« CXCVQL HJll 

datum above surface 16. The parallax of any point in 
the scene is the difference in the coordinates for that 
point for the two images, so that the parallax for each 
of common elevation points A, B, C and D is the same, 
here 40 for the coordinate values shown, while the 
parallax of apex point E is 50. This difference in 
parallax indicates that the elevation of point E is 
different from that of points A, B, C and D and, by 
applying this difference to a conventional parallax- 
based elevation function, the elevation of point E 
relative to surface 16 may be determined. Given the 
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relative elevations of multiple points on a stereo 
xmage of a contoured surface, a model or perspective 
recreation of that surface may be obtained. 

Conventional methodologies for determining 
parallax include those in which an operator looks 
through an optical system to locate elevation points 
and the those which electronically scan a stereo image 
paxr. m the former technique, an operator views the 
overlay of a moveable reticle on a stereo image pair 
through an optical viewfinder and manipulates the 
location of a reticle until, in the operator's best 
estimation, the reticle is coincident with the surface 
of the contour being viewed. This chosen position of 
the reticle is then identified as a surface elevation. 
Obviously, this technique suffers from a variety of 
shortcomings including human judgment, labor intensity 
time consumption and limited data acquisition 
Whatever data is obtained is then used to interpolate 
between points. As a consequence, only a very coarse 
approximation of the surface can be determined. 

In a conventional electronics system, on the 
other hand, such as one employing a flying spot scanner 
to trace successive lines across the images, 
characteristics of the video return signal (typically 
25 delay or shift) are analyzed to determine parallax. 

Still, the resolution is limited by the granularity of 
the scan and the video signal analysis. Moreover, as 
m the operator-based system described previously, the 
number of data points is limited (typically less than 
30 half the number of pixels in the image) so that a 
significant amount of interpolation is required. 

An additional conventional scheme for 
determining parallax subjects each of the images to an 
edge detection mechanism, which attempts to match edge 
35 segments of one image with those of the other, a 

primary shortcoming of this technique is the fact that 
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the parallax can only be calculated for those portions 
of the image which belong to edges (typically on the 
order of only 5-10%) , so that the remainder of the 
image points must be interpolated using these edge 
5 results. 

SUMMARY OF THF. TWROTTffl 

In accordance with the present invention, the 
resolution and. precision limitations of such 
conventional parallax determination techniques are 
10 obviated by a new and improved image analysis mechanism 
which, by means of image pattern correlation, 
determines the parallax for each pixel that is common 
to the images captured by a stereo image pair. In 
particular, the present invention is intended for use 
15 with a digitized image processing system wherein an 

image of a contoured surface of interest is captured as 
a stereo image in the form of first and second 
digitized images. These images are respectively 
representative of first and second views of the 
20 contoured surface as seen from first and second 

respectively different look angles. Each digitized 
image is comprised of a respective array of image 
pixels, so that the stereo image pair is in the form of 
first and second arrays of pixels of a prescribed 
25 spatial resolution (e.g. a 512 x 512 pixel image) and a 
predetermined encoding resolution (e.g. eight or 
sixteen bits per pixel, grey scale) . 

Pursuant to the image analysis process 
conducted in accordance with the present invention, a 
30 sub-array of reference point pixels distributed 

throughout the first image array is defined. Centered 
about each of these reference point pixels is a 
respective first image pattern window. The contents of 
the image pattern windows are correlated with the image 
35 contents of respective first correlation windows, which 
are centered about respective 'tentative' tie point 
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Pixels distributed throughout the second image array in 
the same manner as the sub-array of reference point 
Pixels of the first image array. in order for the 
image contents of a first image pattern window to be 
valid, the contrast within the image pattern contained 
m each fxrst image pattern window must be greater than 
a prescribed contrast value. 

The first correlation windows are 
successively translated or shifted to different 
placements within the first search windows and 
respective correlation values are derived. On the 
basis of these correlations, those pixels of the second 
image array which represent the centers of the first 
correlation windows that produced the highest 
correlation values are identified as 'tie point' 
pixels. The parallax between respective ones of the 
tentative tie point pixels and the correlation- 
identified tie point pixels of the second array is then 
measured. Using these 'tie point '-based parallax 
measurements as starting points, the parallax between 
the remaining pixels of the two image arrays is then 
determined by iteratively conducting search and 
correlate operations similar to those used for locating 
■tie point' parallax. 

More specifically, in accordance with the 
iterative search and correlate process, respective 
boundary windows are defined about the reference point 
pixels of the first image sub-array. Each boundary 
window contains a respective reference point pixel and 
a plurality of neighboring pixels and borders adjacent 
boundary windows of the first image array. The average 
value of the measured parallax for groups of identified 
tie points and their associated tentative tie points is 
measured. Those identified pixels whose parallax 
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differs from the average value- of the measured parallax 
by more than the standard deviation are considered 
invalid and are discarded. 

Using an identified reference point pixel 
within a boundary window of the first image array, for 
which a valid tie point pixel was identified in the 
second image array, as a starting point pixel, the 
image contents of a second image pattern window, which 
contains a pixel adjacent to that starting point pixel 
and a plurality of neighboring pixels, are correlated 
with the image contents of a respective second 
correlation window of the second image array. The 
second correlation window is similarly sized and 
positioned among pixels of the second image array, so 
that it contains a pixel displaced with respect to its 
identified tie point pixel in the same manner that the 
adjacent pixel in the boundary window of the first 
image array is displaced with respect to the starting 
point pixel. The second correlation window is then 
20 successively displaced within a second search window of 
the second array which is centered about the displaced 
Pixel. That position of the second correlation window 
within the second search window which yields the 
highest correlation value with the image pattern 
2 5 contained in the 

— — ^ w wmuow is used to 

identify which pixel of the second array represents a 
location on the contoured surface corresponding to the 
adjacent pixel within the boundary window of the first 
image array, so that the parallax between that adjacent 
30 pixel of the first image array and the thus identified 
pixel of the second search window can be measured. 
This procedure is then iteratively repeated for the 
remaining pixels in each boundary window, always using 
a high confidence identified pixel of the second image 
array as the starting point for the next correlation 
search. 
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After completion of the above procedure, it 
can be expected that there will remain regions of 
pixels surrounding a tentative tie point for which no 
valid tie point was identified, pixels for which 
boundary window search and correlation results were 
insufficient to locate and determine parallax. For 
each of the reference point pixels of the first image 
array for which corresponding valid tie point pixels in 
the second image array were not identified, the 
parallax between pixels contained in the associated 
boundary window and corresponding pixels of the second 
image array is determined in accordance with the 
parallax that has been determined for pixels lying 
along the edge or borders of neighboring boundary 
15 windows that contain valid tie points. 

Specifically, for a respective pixel along 
the edge of a boundary window of the first array for 
which no valid tie point was identified in the second 
array, a corresponding pixel of the second array is 
located in accordance with the geometric relationship 
between that respective pixel and an adjacent pixel of 
a neighboring boundary window the parallax for which 
has been determined. Then, for each respective pixel 
of the second array thus located, a third search window 
is defined so as to contain that pixel and a plurality 
of neighboring pixels. For a respective pixel 
belonging to the boundary window, a third image pattern 
window is defined to contain the respective pixel and a 
plurality of neighboring pixels. The size of the third 
image pattern window is less than the size of the third 
search window. Within the third search window a third 
correlation window, sized in correspondence with the 
third image pattern window of the first image array, is 
defined.. Then, for respectively different locations of 
35 the third correlation window within the third search 
window, the contents of the third correlation window 
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are correlated with the contents of the third image 
pattern window. That pixel among the image pixels of 
the third search window which is the center of the 
third correlation window producing the largest 
5 correlation value is identified as being associated 
with the respective pixel of the boundary window, and 
the parallax of that identified pixel is then measured. 
This process is repeated for each pixel along the edge 
of the boundary window based upon data for pixels along 

10 the edges of bordering boundary windows. The next 

interior lines of the boundary window are processed in 
a similar manner using the edge values, and so on, 
wording into the center of the boundary window, until 
all pixels of the boundary window have been processed. 

15 After this step, the parallax for any 

remaining pixels is determined by scanning the array 
(e.g. from left- to-right and from top- to-bottom) to 
locate a corresponding pixel of the second array in 
accordance with the geometric relationship between that 

20 remaining pixel and an adjacent pixel, the parallax 

between which and a corresponding pixel of the second 
image irray has been determined. For each respective 
corresponding pixel of the second image array thus 
located, a fourth search window, the size of which is 

25 larger than earlier used search windows, is defined to 
contain the corresponding pixel and a plurality of 
neighboring pixels. Centered about a respective 
remaining pixel, a fourth image pattern window is 
defined such that the size of the fourth image pattern 

30 window is increased relative to those previously used. 
Within the fourth search window a fourth correlation 
window sized in correspondence with the fourth 
correlation window is defined. Then, for respectively 
different locations of the fourth correlation window 

35 within the fourth search window, the contents of the 
fourth correlation window are correlated with the 
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contents of the fourth image pattern window, to 
identify about which pixel fourth correlation is 
centered for its largest correlation value. That 
identified pixel is used to determine parallax for the 
5 pixel of interest. 

The images are then again scanned in a 
different direction (e.g. bottom- to- top and right-to- 
left) , and once more in the first direction, each scan 
using correlation and search windows of progressively 

10 larger sizes. At the completion of a this iterative 
search and correlate procedure, the parallax for 
substantially all of the pixels (e.g. on the order of 
97% of the pixels) of that portion of the first and 
second arrays common to each image will have been 

15 determined. 

As a further enhancement of the above 
process, the derived parallax values may be subjected 
to an interpolation function to sub-pixel accuracies. 
The correlation window is translated in integral steps 

20 of one pixel per step within the search window, and 
parallax is determined to sub-pixel accuracies. This 
is accomplished by using a Lagrange polynomial 
interpolation on the values returned by the correlation 
function to determine where the local maxima is 

25 located. This interpolation function calculates the 
value of the correlation function at the integral step 
with the maximum value (the stored parallax result) and 
at two points on either side of the maximum. These 
(three) values are then used in a second order Lagrange 

30 polynomial to interpolate the position (between 

integral pixel steps) at which the local maxima occurs. 
This fractional parallax value is then stored as the 
final result of the correlation process. Once the 
parallax between the first and second image arrays has 

35 been derived, elevations can be computed using a 
conventional parallax-dependent mechanism. 
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RPTEF DESCRIPTION OF THK nP& WTN qp 

Figure 1 diagrammatically illustrates 
respective images of a pyramid as viewed from 
respectively different look angles; 
5 Figure 2 diagrammatically illustrates an 

imagery data processing system in which the processing 
mechanism of the present invention may be incorporated; 

Figure 3 diagrammatically illustrates a pair 
of digitized images of a stereo image pair, the 
10 contents of which are to be subjected to the parallax 
processing mechanism of the present invention; 

Figure 4 diagrammatically illustrates an 
image pattern of a first image array of a stereo image 
pair and associated correlation and search windows of a 
15 second image array of the stereo image pair; 

Figure 5 illustrates a plurality of boundary 
windows of a first image array and associated groups of 
tie points identified within second image array for 
determining average parallax among plural tie points; 
20 Figure 6 shows a respective boundary window 

that contains a starting point pixel and a plurality of 
neighboring pixels; 

Figure 7 shows a matrix of pixels surrounding 
a starting point pixel and arrows which indicate which 
25 previous result is to be used as the initial point for 
the search and correlate process for the next pixel; 

Figure 8 shows an example of the results of 
the neighborhood correlation step, leaving pixel 
locations for which the search and correlation process 
30 failed to derive parallax values; 

Figure 9 shows a boundary window for which no 
associated valid tie point has been was located in the 
second image array and a plurality of bordering 
boundary windows for which valid tie points and at 
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least partially filled pixel values for their 
associated regions in the second image, array were 
correlated during the neighborhood correlation step; 

Figure 10 diagrainmatically illustrates a 
neighborhood recovery order for filling in pixels 
associated with a boundary window for which no 
associated valid tie point has been was located in the 
second image array; 

Figure 11 illustrates how gaps may be left 
along one or more edges of boundary windows, where two 
uncorrelated boundary windows border one another; and 

Figure 12 diagrammatically illustrates how 
the overall image scan proceeds across and down the 
image from left- to-right , top- to-bottom, looking for 
15 uncorrelated pixels. 

Before describing in detail the parallax 
determination mechanism in accordance with the present 
invention, it should be observed that the present 

20 invention resides primarily in a novel structural 

combination of conventional digital imagery processing 
circuits and components and not in the particular 
detailed configurations thereof. Accordingly, the 
structure, control and arrangement of these 

25 conventional circuits and components have been 

illustrated in the drawings by readily understandable 
block diagrams which show only those specific details 
that are pertinent to the present invention, so as not 
to obscure the disclosure with structural details which 

30 will be readily apparent to those skilled in the art 
having the benefit of the description herein. Thus, 
the block diagram illustrations of the Figures do not 
necessarily represent the mechanical structural 
arrangement of the exemplary system, but are primarily 

35 intended to illustrate the major structural components 
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of the system in a convenient functional grouping, 
whereby the present invention may be more readily 
understood. 

Referring now to Figure 2, an imagery data 
5 processing system in which the processing mechanism of 
the present invention may be incorporated is 
diagrammatically shown as comprising an imagery data 
store 100 that contains a plurality of digitally 
encoded stereo image pairs, portions of or the 

10 entireties of which may be selectively accessed by an 
attendant image data processor 102. The digitized 
images in data store 100 may have been obtained from a 
color image transducer device, such as a high 
resolution digital scanner 104 that generates image 

15 characteristic-representative digital signals, 

representative of the image contents of a stereo image 
pair captured on first and second photographic 
recording media 106 onto which a contoured surface of 
interest has been imaged by a stereo image capture 

20 system (e.g. an airborne or spaceborne imaging platform) 
not shown. On the other hand, the digitized image data 
may have been generated directly from digital camera 
units for storage in an associated digital data base 
for subsequent processing by processor 102. It is to 

25 be noted that the present invention is not limited to 
the analysis of stereo image data from a particular 
source. The output of processor 102 may be coupled to 
an attendant display 108 for providing a presentation 
of perspective image that has been generated using 

30 elevation data derived in accordance with the parallax 
determined by the iterative search and correlate 
process of the present invention to be described in 
detail below. 

As pointed out briefly above, the present 

35 invention uses a search and correlate digitized image 
processing routine to determine the parallax for each 
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pixel that is common to the images captured by a stereo 
image pair. Figure 3 diagrammatically illustrates a 
pair of digitized images 21 and 22 of a stereo image 
pair, the contents of which are to be subjected to the 
parallax processing mechanism of the present invention. 
As a non- limitative example, each of images 21 and 22 
may comprise an array of 512 x 512 pixels the grey 
scale values of which have been digitally encoded to an 
eight or sixteen bit resolution. The images themselves 
may be output directly by high resolution digital 
camera units, or they may be derived by digitally 
scanning first and second stereo images that have been 
captured by an airborne/ spacebome imaging system used 
for terrain mapping, for example. For purposes of the 
present description, any non-common portions of the 
images will be ignored. 

Pursuant to the image analysis process of the 
present invention, computation complexity is 
substantially reduced by an iterative process which 
steps through a sequence of correlation searches, each 
of which is initially referenced to what has been 
determined to be valid data, so that the data 
processing time to locate further data points can be 
reduced substantially. 



TIE POINT CORRELATION: STEP 1 

The process begins by conducting a coarse 
parallax determination of the overall image by means of 
a sub-array of preestablished reference or 'starting 
30 points' distributed throughout the image. For this 
purpose, one of the images of the image pair, image 
array 21 for example, is subdivided into a sub-array of 
reference or • starting- , points 25 evenly distributed 
throughout the image array. For the example of an 
image containing a (512 x 512) array of pixels, image 
21 may be subdivided into a sub-array of (32 x 32) 
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starting points 25, having a spatial periodicity of 16 
pixels, so as to provide ample room for a reasonably 
sized correlation window to be framed about each 
starting point without overlapping that of an adjacent 
5 starting point. The sub-array of starting points 25 is 
referenced to a coordinate system commonly employed for 
each image, so that for the second image 22, a like 
sub-array of tentative 'tie' points 26 may be evenly 
distributed among the pixels of that image. Namely, 
10 points 26 represent locations on the surface contour 
that are to be 'tied' to the starting points, for the 
purpose of establishing a base parallax reference upon 
which the remainder of the computations will be depend. 
Once the locations of these 'tie 1 points have been 
15 determined parallax measurements for the remaining 
pixels of the image are conducted in local 
neighborhoods associated with each tie point, using its 
parallax as a base parallax value for that 
neighborhood. Of course, due to parallax between the 
20 two images whatever pixels have been initially chosen 
as tentative ■ tie point ■ pixels in image array 22 will 
be necessarily offset from the coordinate locations of 
starting point pixels 25 of image array 21. 

With the sub-array of starting points 25 
25 defined within the first image array 21, a respective 
first image pattern window IP1 is formed about each 
respective starting point pixel 25, as diagrammatically 
illustrated in Figure 4. As shown in Figure 4, a 
respective first image pattern window IP1 contains (is 
30 centered about) a starting point pixel 25, and 

encompasses a plurality of neighboring pixels 31. In 
the illustrated example, image pattern window IP1 is a 
square (5x5) matrix of pixels, containing a pattern 
of relatively darker pixels 33 bordered by adjacent 
35 relatively lighter pixels 35. It should be observed, 
however, that the size of the image pattern matrix is 
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not limited to this or any other particular size. In 
an actual prototype example, the image pattern window 
employed an (11 x 11) matrix; the (5 x 5) matrix shown 
here is for purposes of simplifying the drawing. 

Referring again to Figure 3, associated with 
each starting point-centered first image pattern window 
IP1 of first image 21 are a. plurality of first search 
windows SW1 distributed among the second array 22, such 
that each first search window SW1 encompasses a 
respectively different plurality of pixels that 
contains a tentative 'tie' point pixel 26. Because the 
sub-array of tentative tie point pixels 26 is 
referenced to the same coordinate system employed for 
starting point pixels 25 in the first image array 21, 
tentative tie point pixels 26 are effectively evenly' 
distributed among the pixels of the second image, such 
that, within image 22, a respective tentative tie point 
pixel 26 is located geometrically proximate to 'the 
location of a respective starting point pixel 25 within 
20 image 21. 

As shown in Figure 4, the size of a first 
search window SW1 is larger than the first correlation 
window CW1 and is centered about a tentative tie point 
pixel 26. While an actual prototype example employed a 
(35 x 15) matrix, the (9 x 9) matrix shown in Figure 4 
is for purposes of simplifying the drawing. 
Correlation window CW1 is the same size as first image 
pattern window IP1. To provide for correlation 
computational efficiency, the size differential between 
a search window and a correlation window is preferably 
such that the number of shifts or translation of a 
correlation window within the search window is limited 
to a reasonable number (e.g. twenty- five for the 
illustrated example, although the above-referenced 
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prototype example employed 125 shifts) . What is 
important is making sure that the correct match is 
located within the search window. 

The search and correlate process in 
accordance with the present invention proceeds on the 
assumption that the contour of the surface of interest 
does not have large discontinuities, so that the 
parallax measurement from one pixel to another will 
also not be discontinuous. As a consequence, the 
parallax offset for one pixel can be used as an initial 
'guess' or starting point for the parallax of a 
neighboring pixel. Centering the search window on this 
starting point will likely cause the pattern which most, 
closely matches the image contents of the correlation 
pattern to be nearby, thereby allowing the search 
window to be kept to a reasonably small size. In the 
course of locating which pixels of the second array 
correspond to or 'tie' to starting points of the first 
image', however, since nothing is initially known about 
the parallax in this area of the image, the size of the 
first search window must be relatively large. This is 
particularly true since subsequent correlations will be 
based on the locations of the initial tie points in the 
second image array. Thus the locations of the tie 
25 points in the second image must be correct. A larger 
area first search, window SW1, having a high signal-to- 
noise ratio, is used during this step of the process. 
The large signal-to-noise ratio is due to the large 
image pattern window employed (which, in turn, leads to 
30 a larger search window) . 

As shown in Figure 4, first correlation 
window CW1 is initially centered within the search 
window SW1, so that it is centered about 'tentative' 
tie point pixel 26. For this and subsequent locations 
35 of first correlation window CW1 within first search 
window SW1, the image contents of correlation window 
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CW1 are correlated with the contents of the first image 
pattern window IP1. while a number of image- 
correlation functions that operate on two image can be 
used, such as the correlation intensity coefficient, 
5 the phase correlation coefficient and the Laplace 

coefficient, a preferred function is the normal product 
function since it has good immunity to low signal-to- 
noise ratios and requires significantly less time to 
compute than most of the other functions. 
10 The normal product moment correlation function is: 

N N 

Uljfl [tI l (X i + **. Yj + Ay) - ^Hiaffi. Yj) . J j]2 
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The value of this function is between 0.0 and 
1-0, with larger values indicating better matches 
between the first pattern window and the first 
correlation window. This function is repeatedly 
20 evaluated as the correlation window is moved within the 
search window, with the final result being the location 
at which the correlation function has a maximum value. 

Because the correlation process operates on 
image patterns, it is necessary that the pattern have 
25 sufficient definition or contrast, so that the 

likelihood is high that the results of the correlation 
will correctly identify the location of a contour point 
of interest. For this purpose, prior to stepping the 
correlation window through its search window, the 
contents of the image pattern are evaluated for 
sufficient contrast to form a pattern that can be 
recognized. An area of desert sand or open water, for 
example, may contain little, if any, contrast, and 
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therefore be unsuitable for use as a correlation 
reference pattern. If the variance in the pattern 
window is less than a minimum specified threshold, 
namely the contents of image pattern window IP1 have 
5 less than a prescribed amount of contrast, then no 
attempt is initially made to process that image 
pattern. 

If there is sufficient contrast within the 
image pattern window, then the process proceeds to 

10 shift or translate first correlation window CW1 through 
search window SW1. For each position of first 
correlation window CW1 within search window SW1, the 
image contents of correlation window CW1 are correlated 
with the image contents of its associated first image 

15 pattern window IP1 using the normal product moment 
function, set forth above, thereby deriving a 
correlation value for each position of the correlation 
window. For that location of correlation window CW1 
producing the largest correlation output value, the 

20 center pixel of the correlation window CW1, shown at 
26' in Figure 4, is identified as the tie point of 
interest. However, again, as in the case of the 
requirement of minimum contrast in the image pattern 
window, the maximum correlation value must be above a 

25 minimum threshold, in order for the correlation output 
to be acceptable. If the correlation is less than the 
this threshold, then no attempt is made to process the 
tie point. The parallax of identified tie points 26' 
is then measured as the offset from the tentative 

30 points 26. 

Before accepting the parallax measurements 
for the identified tie point pixels 26* of the second 
image array, the parallax measurement values resulting 
from the tie point correlation process are subjected to 
35 a spatial offset error detection function which 

analyzes the identified locations of tie point pixels 



WO 92/06444 



PCT/US91/06908 



10 



15 



20 



-18- 

26' associated with starting point pixels 25 of a 
plurality or group of boundary windows BW that are 
defined about starting point pixels 25 of the first 
image array, as illustrated in Figure 5. As shown 
therein, each boundary window BW contains a respective 
starting point pixel 25 and a plurality of neighboring 
Pixels and borders the boundary windows of adjacent tie 
points. The average value of the measured parallax for 
groups or regions of first search windows SW1 within 
second image array 22 is measured and those identified 
tie point pixels 26' whose parallax differs from the 
average value of the measured parallax by more than the 
standard deviation are discarded. The parallax for 
each of the non-discarded tie points 26' is then 
employed to determine the parallax between remaining 
locations on the contoured surface as seen in the first 
digitized image 21 and corresponding locations on the 
contoured surface as seen in the second digitized 
image 22. 



NEIGHBORHOOD CORRELATION: STEP 2 

Neighborhood correlation operates upon the 
each of the pixels within the respective boundary 
windows BW for which valid tie points were identified 

25 during the tie point correlation sequence. Because a 

large number of pixels are to be processed, the size of 
the correlation window should be smaller than in the 
tie point correlation, while such a reduction will 
lower the signal-to-noise ratio of the function, since 

30 the search window is (indirectly) based upon a highly 
confident result, the potential for error is not 
significantly increased. Moreover, the size of the 
search window may also be reduced, since the parallax 
will not be discontinuous and the search window can be 

35 centered on the result from an adjacent pixel. 

Employing such smaller sized correlation and search 
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windows dramatically reduces execution time. for this 
portion of the process. As an illustrative example, 
the correlation window may encompass a (7x7) matrix 
and the search window may cover a (9x9) matrix. 
5 As described above and as shown in Figure 6, 

a respective boundary window BW contains a starting 
point pixel 25 and a plurality of neighboring pixels 
41-1 ... 41-n and is sized such that adjacent boundary 
windows border one another. The neighborhood search 
10 and correlate prbcess begins at a pixel, such as pixel 
41-1, adjacent to the starting point pixel 25 and 
operates to locate a corresponding pixel within the 
second array using, as an initial reference, the 
parallax that has been determined for tie point 26'. 
15 For this Purpose, the position of pixel 41-1 

relative to starting point 25 is used to locate a pixel 
41-1* within second image array 22 that is similarly 
located relative to tie point 26'. in the example 
shown' in Figure 6, pixel 41-1 is to the immediate left 
20 of starting point pixel 25. Consequently, a second 

search window SW2 is centered about pixel 41-1- within 
second image array 22 which is to the immediate left of 
tie point pixel 26'., A second image pattern window IP2 
is centered about pixel 41-1 within its boundary window 
25 BW of the first image array 21 and its contrast is 
measured as in the case of the image pattern window 
used for tie point correlation, described above. 
Assuming that image pattern window IP2 contains a valid 
pattern, namely, there is sufficient contrast within 
the image pattern window, then its contents are 
correlated with those of a second correlation window 
CW2 within second search window, as the process 
successively translates the second correlation window 
CW2 through second search window SW2 . 



30 



WO 92/06444 



PCI7US91/06908 



-20- 



10 



15 



As in the tie point search, for each position 
of second correlation window CW2 within second search 
window SW2, the image contents of correlation window 
CW2 are correlated with the image contents of its 
associated second image pattern window IP2 using the 
normal product moment function, set forth above, 
thereby deriving a correlation value for each position 
of the correlation window. For that location of 
correlation window CW2 within search window SW2 
producing the largest correlation output value, the 
center pixel of the correlation window CW2, shown in 
Figure 6 at pixel 41-1- , is identified. Again, as in 
the case of the tie point search, the maximum 
correlation value must be above a minimum threshold, in 
order to the correlation output to be acceptable. If 
the correlation is less than this threshold, then no 
attempt is made to process the point. The parallax of 
identified point 41-1- is then measured as the -of f set 
of identified point 41-1- from pixel point 26 minus the 
offset of point 41-1 from pixel point 25. This 
procedure is then iteratively repeated for the 
remaining pixels in each boundary window, always using 
a high confidence identified pixel of the second image 
array as the starting point for centering the search 
25 window for the next correlation search. 

As a reduced complexity illustration, Figure 
7 shows a boundary window BW comprised of a (5x5) 
matrix of pixels 41-1 ... 41-24 surrounding a starting 
point pixel 25. The arrows indicate which previous 
30 result is to be used as the initial point for the next 
pixel. Thus, for the case of the immediately previous 
result for pixel 41-1, the next pixel to be used to 
center image pattern window IP2 is shown as pixel 41-2, 
which is to immediate left of pixel 41-1 within 
35 boundary window BW. Therefore, second search window 
SW2 is now centered about pixel 41-2 1 within second 
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image array 22 which is to the immediate left of the 
most recently identified pixel 41-1". With second 
image pattern window IP2 centered about pixel 41-2, the 
contrast of its image pattern is measured. Assuming 
5 that image pattern window IP2 contains a valid pattern, 
then its pattern is correlated with the contents of 
second correlation window CW2 within second search 
window, as second correlation window CW2 is 
successively translated through second search window 

10 SW2 now- centered about pixel 41-2' . 

As in the previous search, for each position 
of second correlation window CW2 within second search 
window SW2, the image contents of correlation window 
CW2 are correlated with the image contents of its 

15 associated second image pattern window IP2 using the 
normal product moment function, set forth above, 
thereby deriving a correlation value for each position 
of the correlation window. For that location of 
correlation window CW2 within search window SW2 

20 producing the largest correlation output value, the 
center pixel of the correlation window CW2, shown in 
Figure 6 at pixel 41-2- , is identified. Again, the 
maximum correlation value must be above a minimum 
threshold, in order to the correlation output to be 

25 acceptable. If the correlation is less than this 
threshold, then no attempt is made to process the 
point. The parallax of identified point 41-2' is then 
measured as the offset of identified point 41-2" from 
pixel point 26 minus the offset of point 41-2 from 

30 pixel point 25 and the process proceeds to pixel 41-3, 
shown in Figure 7 as being directly above starting 
point pixel 25, and search window SW2 is 
correspondingly centered at pixel 41-3 • directly above 
tie point pixel 26*. 
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It should be observed that centering each 
neighborhood of pixels about its tie point reduces the 
possibility of introducing errors by reducing the 
number of steps between the point for which the search 
5 and correlated sequence is to be conducted and the 
•high confidence' placed on the location of the tie 
point, thereby minimizing the likelihood that the 
process will 'drift* away from the correct result. 

As described above, neighborhood correlation 

10 operates, upon the' each of the pixels within respective 
boundary windows BW for which valid tie points were 
identified during the tie point correlation sequence. 
In some instances, however, the first image array will 
contain boundary windows for which tie points within 

15 the second image array were not found during tie point 
correlation. These initially discarded tie points will 
leave associated areas or regions of pixels within 
image array 22 without parallax values, as illustrated 
at re'gion 51, for example* shown in Figure 8. In 

20 addition, during the neighborhood correlation step, it 
can be expected that there will be pixel locations for 
which the correlation process failed to derive parallax 
values, as shown at 53 in region 55 in Figure 8. The 
neighborhood recovery stage serves to correlate pixels 

25 in regions where no valid tie point was identified 
using correlated pixel values correlated from pixels 
along the edges of bordering boundary windows. 
Following this step, the process proceeds to 
iteratively fill in values for individual pixels 

30 remaining uncorrelated from the previous steps. 

NEIGHBORHOOD RECOVERY: STEP 3 

More particularly, for each starting point 
pixel 25 of first image array 21 for which no 
35 corresponding tie point pixel 26' within the second 
image array 22 was identified during the tie point 
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correlation step, pixel data along the edges of 
bordering boundary windows is employed to identify 
pixels along adjacent edges of the region of interest. 
These values, in turn, are then used to generate a more 
5 interior group of pixels, and so on until the tentative 
tie point is reached. This process is conducted, to 
the extent possible, from each edge of the boundary 
window. 

As a simplified example, Figure 9 shows 

10 boundary window 51 for which no associated valid tie 
point was located in second image array 22, and a 
plurality of bordering boundary windows 52, 54, 55 and 
56 for which valid tie points and at least partially 
filled pixel values for their associated regions in the 

15 second image array were correlated during the 

neighborhood correlation step. In order to derive 
pixel data for those pixel points along its border with 
boundary window, a respective 'nearest' edge pixel, 
such as pixel 52-1, of bordering boundary window 52, 

20 for which parallax data in the second image array has 
been derived, is used as a starting reference for a 
first edge pixel 51-1 of boundary window 51. Namely, 
as in the previous correlation steps, the position of 
pixel 51-1 relative to an already successfully 

25 correlated pixel location, here pixel 52-1, is used to 
locate a pixel 51-1', that is similarly located 
relative to pixel 52-1", as shown in second image array 
22 of Figure 9. In Figure 9, pixel 51-1 is immediately 
beneath pixel 52-1. Consequently, a third search 

30 window SW3 is centered about pixel 51-1 1 of second 

image array 22, pixel 51-1 1 being immediately beneath 
pixel location 52-1*. 

A third image pattern window IP3 is centered 
about pixel 51-1 of the first image array 21 and its 

35 contrast is measured as in the case of image pattern 

window used for tie point correlation, described above. 
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Assuming that third image pattern window IP3 contains a 
valid. pattern, namely, there is sufficient contrast 
within the image pattern window, then its contents are 
correlated with those of a third correlation window CW3 
5 within a third search window SW3, as the process 

successively translates the third correlation window 
CW3 through third search window SW3 . As in the 
neighborhood correlation search, for each position of 
third correlation window CW3 within third search window 

10 SW2, the image contents of third correlation window CW3 
are correlated with the image contents of its 
associated third image pattern window IP3 using the 
above-referenced normal product moment function, 
thereby deriving a correlation value for each position 

15 of correlation window CW3 within search window SW3 . 
For that location of third correlation window CW3 
within third search window SW3 producing the largest 
correlation output value, the center pixel of ' 
correlation window CW3 is identified. Again, as in 

20 previous searches, the maximum correlation value must 
be above a minimum threshold, in order to the 
correlation output to be acceptable. If the 
correlation is less than this threshold, then no 
attempt is made to process the point. The parallax of 

25 the identified point 51-1- is then measured as the 
offset of identified point 51-1- from pixel point 26 
minus the offset of point 51-1 from pixel point 25. 
This procedure is then repeated for remaining pixels 
51-2 ... 51-4 along the edge in boundary window 51, 

30 always using a nearest previously identified pixel of 
the adjacent boundary window 52 as the starting point, 
using the neighborhood recovery order diagrammatically 
illustrated in Figure 10. 

In accordance with the neighborhood recovery 

35 order shown in Figure 10, when each of edge pixels 51-1 
. . . 51-4 has been correlated, the process uses two of 



WO 92/06444 



PCT/US9 1/06908 



-25- 

theses pixels, specifically pixel 51-2 and 51-3 to 
correlate values for immediately adjacent pixels 51-7 
and 51-8, respectively, in the next interior pixel line 
containing pixels 51-6 ... 51-10. However, pixels 51- 
5 6, 51-9 and 51-10 are not processed on the basis of the 
pixels 51-1, 51-4 and 51-5, respectively. Pixels 51-6 
and 51-10 have nearest previously correlated pixels 54- 
10 and 55-6, respectively. Pixel 51-9 is referenced to 
pixel 51-10 to provide a reasonably symmetrical pixel 

10 correlation order. With the remaining pixels being 
processed in the order shown in Figure 10, then, for 
the next (interior-most) line, only pixel 51-13 is 
processed based upon a previously processed pixel 
within boundary window 51, here based upon nearest 

15 neighboring pixel 51-8. " 

To the extent that each uncorrelated boundary 
window has neighboring boundary windows containing 
correlated pixels, the process described above is 
carried out using the neighborhood recovery order shown 

20 in Figure 10. Where two uncorrelated boundary windows 
border one another, gaps will be left along one or more 
edges of the boundary windows, as shown at 71 and 73 in 
Figure 11. At this point in the process, local groups 
or neighborhoods of high confidence pixel locations 

25 that may be used as reference points for further 

processing have been effectively exhausted. To fill in 
remaining pixels in the image, an iterative sequence of 
overall image scans is conducted, using, as a starting 
point for any uncorrelated pixel, any correlated 

30 nearest neighbor pixel. In addition as successive 

scans of the image are executed, the size of both the 
search window and the correlation window are increased. 

OVERALL IMAGE SCAN: STEP 4 
35 More particularly, with reference to Figure 12, the 

overall image scan proceeds across and down the image 
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from left-to-right, top- to-bottom, looking for 
uncorrected pixels. When any uncorrelated pixel is 
encountered, such as pixel 81, any nearest neighbor 
pixel for which a valid correlation has been obtained 
5 is used as the starting point of the search and 
correlate process. Namely, as in the previous 
correlation steps, the position of pixel 81 relative to 
an already successfully correlated pixel location, such 
as pixel 83 to the immediate left of pixel 81, is used 
10 to locate a pixel 81' within second image array 22, 
that is similarly located relative to pixel 83-, as 
shown in second image array 22 of Figure 12. In Figure 
12, pixel 81 is to the immediate right of valid pixel 
83. Consequently, a fourth search window SW4 is 
15 centered about pixel 81' within second image array 22, 
pixel 81' being to the immediate right of location 83". 

A fourth image pattern window IP4, which is 
preferably the same size as third image pattern" window 
(e.g. a 7 x 7 matrix) is centered about pixel 81 within 
first image array 21 and its contrast is measured, as 
described above. Assuming that fourth image pattern 
window IP4 contains sufficient contrast, then its 
contents are correlated with those of a fourth 
correlation window CW4 within a fourth search window 
25 SW4, as the process successively translates the fourth 
correlation window CW4 through fourth search window 
SW4. Fourth search window SW4 is the same size as the 
third search window and may encompass, for the 
illustrated example, a (9x9) matrix. As in the 
30 previous searches, for each position of fourth 

correlation window CW4 within fourth search window SW4, 
the image contents of fourth correlation window CW4 are 
correlated with the image contents of its associated 
fourth image pattern window IP4 using the above- 
35 referenced normal product moment function, thereby 
deriving a correlation value for each position of 
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correlation window CW4 within the search window. 
Again, for that location of the correlation window 
within the search window producing the largest 
correlation output value, the center pixel of the 
5 correlation window is identified. Also, as in previous 
searches, the maximum correlation value must be above a 
minimum threshold, in order to the correlation output 
to be acceptable. If the correlation is less than this 
threshold, then no attempt is made to process the 

10 point. The parallax of the identified point 81" is 
then measured as the offset of identified point 81" 
from pixel point 26 minus the offset of point 81 from 
pixel point 25. This procedure is then repeated for 
remaining uncorrelated pixels of the image during this 

15 first scan, always using a nearest previously 
identified pixel as the starting point. 

After a first pass through the image in a 
left-to-right, top- to-bottom scan of the image," filling 
in, to the extent possible, uncorrelated pixels, the 

20 image is again retraced, but following a different 

traverse of the image, e.g. right- to-left , bottom- to- 
top, and using increased search and correlation window 
sizes, such as an (11 x 11) correlation matrix and a 
(13 x 13) search window. Namely, successive rescans of 

25 the image are conducted from opposite directions and 
with increasingly larger correlation and search 
windows, such that the maximum change in parallax from 
the neighboring pixel point remains constant. Thus, as 
in the case of the foregoing examples, a larger 

30 correlation window (15 x 15) would have an (17 x 17) 
search window and so on. It turns out that after a 
sequence of three overall scans of the image, parallax 
values for a very high percentage of those pixels 
having significant contrast will have been determined. 
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In order to detect and correct for local 
inconsistencies and discontinuities in the parallax 
data derived during the overall scan process and 
previous search and correlation steps, between the 
bottom-to-top, right-to-left return scan and a 
following repeat of the left- to- right , top-to-bottom 
scan (using a (15 x 15) correlation window and a (17 x 
17) search window), the parallax values of each 
correlated pixel and those of neighboring pixels whose 
parallax" differ by two or more are subjected to the 
previously described standard deviation comparison used 
for tie points. Any parallax value that differs by 
more than one standard deviation is discarded. Then, 
in the following repeat of the left-to-right, top- to- 
bottom scan (using a (15 x 15) correlation window and a 
(17 x 17) search window any such discarded pixels will 
have the opportunity to be recorrelated. Indeed, since 
this scan employs a very large correlation window 
having high signal-to-noise ratio, the recorrelated 
pixels are much more likely to be correct. 

SUB- PIXEL INTERPOLATION 

As a further enhancement of the above 
process, the derived parallax values may be subjected 
to an interpolation function to sub-pixel accuracies. 
The correlation window can be moved only in integral 
steps of one pixel per step within the search window, 
and because a difference in parallax of one pixel may 
result in a substantial change in elevation, it is 
30 preferred that the parallax be determined to sub-pixel 
accuracies. This is accomplished by using a Lagrange 
polynomial interpolation on the values returned by the 
correlation function to determine where the local 
maxima is located. This interpolation function 
35 calculates the value of the correlation function at the 
integral step with the maximum value (the stored 
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parallax result) and at two points on either side of 
the maximum. These (three) values are then used in a 
second order Lagrange polynomial to interpolate the 
position (between integral pixel steps) at which the 
local maxima occurs. This fractional parallax value is 
then stored as the final result of the correlation 
process. it should be observed that this interpolation 
can only be executed on points which were successfully 
correlated in a preceding step; points which remain 
uncorrected will not be affected by this process. 

Once the parallax between the first and 
second image arrays has been derived, elevations can be 
computed using a conventional parallax-dependent 
mechanism. One example of a suitable mechanism for 
15 this purpose is the equation: 

h A = H - (B*f)/P A 
where H is the height of the image sensor above the 
surface, B is the distance between the positions when 
the stereo image pair was captured, f is the effective 
focal length of the sensor and P A is the parallax. For 
points which do not have a computed parallax value, 
elevation is interpolated from neighboring points. The 
interpolation mechanism calculates a weighted average 
of known elevation points which appear above, below, to 
the left and to the right of the point of interest and 
employs this value in lieu of a parallax-based 
elevation value. 

As will be appreciated from the foregoing 
description, the resolution and precision limitations 
of conventional parallax determination techniques are 
obviated in accordance with the digitized image 
processing mechanism of the present invention which, by 
means of an iterative image pattern search and 
correlation process, which is able to rapidly determine 
35 the parallax for each pixel that is common to the 

images captured by a stereo image pair. By the initial 
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use of a dispersed sub-array of reference point pixels 
distributed throughout a first image array, a similarly 
dispersed sub-array of tie points, to be used as 
starting points for subsequent, more densely populated 
search and correlation steps, can be rapidly obtained 
for the second image array. The derived parallax 
values may then be subjected to an interpolation 
function to sub-pixel accuracies. Once the parallax 
between the first and second image arrays has been 
derived,- elevations can be computed using a 
conventional parallax-dependent mechanism. 

While I have shown and described an 
embodiment in accordance with the present invention, it 
is to be understood that the same is not limited 
thereto but is. susceptible to numerous changes and 
modifications as known to a person skilled in the art, 
and I therefore do not wish to be limited to the 
details shown and described herein but intend to cover 
all such changes and modifications as are obvious to 
one of ordinary skill in the art. 
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WHAT Tfi PT.ATMPn 

1, For use with a digitized image 
processing system wherein first and second digitized 
images respectively representative of first and second 
5 views of a contoured surface as seen from first and 
second respectively different look angles are 
generated, said first and second digitized images 
containing respective first and second. arrays of image 
pixels, a method of determining parallax between 

10 locatiqns on said contoured surface as seen in said 
first digitized image and corresponding locations on 
said contoured surface as seen in said second digitized 
image, comprising the steps of: 

(a) correlating the image contents of a 

15 plurality of first image pattern windows distributed 
throughout said first array, each first image pattern 
window containing a respective sub-array of pixels of 
said first array, with the image contents of respective 
first correlation windows, each first correlation 

20 window containing a respective sub-array of pixels of 
said second array, for different placements of said 
first correlation windows within a plurality of first 
search windows distributed throughout said second 
array; 

25 (b) on the basis of correlations carried out 

in step (a) , identifying which pixels of said second 
array represent locations on said contoured surface 
that correspond to prescribed pixels of said first 
array; 

30 (c) determining the parallax between 

respective ones of the prescribed pixels of said first 
array and associated identified pixels of said second 
array; and 
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(d) determining the parallax between 
locations on said contoured surface as seen in said 
first digitized image and corresponding locations on 
said contoured surface as seen in said second digitized 
image, in accordance with the parallax determined in 
step (c) . 

2. A method according to claim 1, wherein 
step (d) comprises the steps of: 

<dl) defining a boundary window about each of 
the prescribed pixels of said first array, such that a 
respective boundary window contains a respective 
prescribed pixel and a plurality of neighboring pixels 
and such that adjacent boundary windows border one 
another ; and 

<d2) determining the parallax between pixels 
contained in respective boundary windows and 
corresponding pixels of said second array, in 
accordance with the parallax determined in step (c) . 

3. A method according to claim 2, wherein 
step (d2) comprises the steps of: 

(d2-l) using a respective pixel of said 
second array identified in step (b) as a reference 
pixel, correlating the image contents of a second image 
pattern window, which contains a pixel adjacent to a 
25 prescribed pixel and a plurality of neighboring pixels 
of said first array, with the image contents of a 
respective second correlation window, said second 
correlation window containing a pixel displaced with 
respect to said reference pixel of said second array, 
in the same manner that said adjacent pixel is 
displaced with respect to said prescribed pixel of said 
first array, and a plurality of neighboring pixels of 
said second array, for different placements of said 
second correlation window within a second search window 
35 of said second array; 
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(d2-2) on the basis of correlation carried 
out in step (d2-l) , identifying which pixel of said 
second array represents a location on said contoured 
surface that corresponds to said adjacent pixel within 
5 said boundary window of said first array; 

(d2-3) determining the parallax between said 
adjacent pixel of said first array and the pixel of 
said second search window identified in step (d2-l) ; 
and 

10 (d2-4) using a respective pixel of said 

second array identified in step (d2-2) as a reference 
pixel, repeating steps (d2-l) - (d2-3) for the 
remaining pixels of each respective boundary window. 

4. A method according to claim 2, wherein 
15 step (a) comprises measuring the contrast within said 

first image pattern window, and wherein steps (b) - (d) 
are carried out with respect to said first image 
pattern window in response to the value of the measured 
contrast being greater than a prescribed contrast 
20 value* 

5 . A method according to claim 4 , wherein 
step (c) includes the steps of measuring the parallax 
between respective identified pixels of said first 
search windows and associated prescribed pixels of said 

25 first array, determining the average value of said 
measured parallax for a plurality of first search 
windows, and discarding those identified pixels whose 
parallax differs from said average value of said 
measured parallax by a prescribed difference, 

30 6. A method according to claim 5, wherein 

said prescribed difference corresponds to the standard 
deviation. 
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7. A method according to claim 3, further 
including the step of: 

(e) for each of the prescribed pixels of 
said first array for which corresponding pixels of said 
5 second array were not identified in step (b) . each of 
such prescribed pixels having an associated boundary 
window as defined in step (dl) , determining the 
parallax between pixels contained in said associated 
boundary window and corresponding pixels of said second 
10 array, in accordance with the parallax determined in 

step (d2^4) between locations on said contoured surface 
as seen in said first digitized image and corresponding 
locations on said contoured surface as seen in said 
second digitized image i 

15 8. a method according to claim 7, wherein 

step (e) comprises the steps of: 

(el) for a respective pixel belonging to an 
associated boundary window, locating a corresponding 
pixel of said second array in accordance with the 

20 geometric relationship between said respective pixel 

and a neighboring pixel the parallax between which and 
a corresponding pixel of said second array has been 
determined; 

(e2) or each respective pixel of said second 
25 array located in step (el), defining a third search 
window containing that pixel and a plurality of 
neighboring pixels; 

(e3) for a respective pixel belonging to said 
associated boundary window, defining a third image 
pattern window containing said respective pixel and a 
plurality of neighboring pixels, such that the size of 
said third image pattern window is less than the size 
of said third search window; 
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(e4) defining, within said third search 
window, a third correlation window sized in 
correspondence with said third image pattern 
correlation window; 

5 (e5) for respectively different locations of 

said third correlation window within said third search 
window, correlating the contents of said third 
correlation window with the contents of said third 
image pattern window; 

10 < e6 > specifying which pixels among the image 

pixels of said third search window are associated with 
locations on said contoured surface represented by 
pixels of said associated boundary window, in 
accordance with the correlation carried out in step 

15 (e5); and 

(e7) determining the parallax between 
locations on said contoured surface as seen in said 
first digitized image and corresponding locations on 
said contoured surface as seen in said second digitized 
20 image in accordance with separations between respective 
pixels of said associated boundary window and image 
pixels of said third search window specified in step 
<e6) . 

9. A method according to claim 8, wherein 
25 step (el) comprises, for a respective pixel belonging 

tO an associated boundarv win^nu 1 ocat-in.-r s 

corresponding pixel of said second array in accordance 
with the geometric relationship between said respective 
pixel and a pixel of a neighboring boundary window. 
30 10. A method according to claim 7, further 

including the step of: 

(f) for each of remaining pixels of said 
first array, for which parallax with pixels of said 
second array has not been determined through step (e) , 
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determining the parallax between such pixels and 
corresponding pixels of said second array in accordance 
with the parallax by the steps of 

(fl) for a respective remaining pixel, 
locating a corresponding pixel of said second array in 
accordance with the geometric relationship between said 
respective remaining pixel and an adjacent pixel the 
parallax between which and a corresponding pixel of 
said second array has been determined; 

(f2) for each respective corresponding pixel 
of said second array located in step (fl), defining a 
fourth search window containing said respective 
corresponding pixel and a plurality of neighboring 
pixels; 

15 < f3 > for a respective remaining pixel, 

defining a fourth image pattern window containing said 
respective remaining pixel and a plurality of 
neighboring pixels, such that the size of said "fourth 
image pattern window is less than the size of said 
20 fourth search window; 

(f4) defining, within said fourth search 
window, a fourth correlation window sized in 
correspondence with said fourth image pattern window; 

(f5) for respectively different locations of 
25 said fourth correlation window within said fourth 

search window, correlating the contents of said fourth 
correlation window with the contents of said fourth 
image pattern window; 

(f6) specifying which pixels among the image 
pixels of said fourth search window are associated with 
locations on said contoured surface represented by 
remaining pixels of said first array, in accordance 
with the correlation carried out in step (f5) ; and 
(f7) determining the parallax between . 
35 locations on said contoured surface as seen in said 
first image and corresponding locations on said 
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contoured surface as seen in said second digitized 
image in accordance with separations between -respective 
pixels of said first array and image pixels of said 
fourth search window specified in step (f6). 
5 11. A method according to claim 10, further 

comprising repeating steps (fl) - (f7) using 
correlation and search windows of progressively larger 
sizes. 

12. A method according to claim 10, further 
10 comprising repeating steps (fl) - (f7) using a 

different traverse of said first and second pixel 
arrays . 

13. A method according to claim 1, further 
including the step of (e) determining parallax for sub- 

15 pixels of said image by interpolating the parallax 
values derived in step (d) . 

14. A method according to 13, wherein step 
(e) comprises determining a correlation value for an 
integral pixel step at a maximum value of parallax and 

20 at two points on either side of the maximum, and 

processing the correlation values to interpolate the 
position between integral pixel steps at which the 
local maxima occurs . 

15. A method according to claim 1, further 
25 including the step of (e) determining elevations of 

locations on said contoured surface in accordance with 
the parallax determined in step (d) . 

16. For use with a digitized image 
processing system wherein first and second digitized 

30 images respectively representative of first and second 
views of a contoured surface as seen from first and 
second respectively different look angles are 
generated, said first and second digitized images 
containing respective first and second arrays of image 
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pixels, a method of determining parallax between 
locations on said contoured surface as seen in said 
first image and locations on said contoured surface as 
seen in said second image, comprising the steps of: 
5 a) defining a first sub-array of pixels of 

said first image of said contoured surface, said first 
sub-array of pixels encompassing that portion of said 
contoured surface seen in both of said first and second 
images ; 

10 b) defining a first image pattern window 

about each of the pixels of said first sub-array, such 
that a respective first image pattern window contains a 
respective pixel of said first sub-array and a 
plurality of neighboring pixels; 

15 c > defining a plurality of first search 

windows among said second array of image pixels, each 
of said first search windows encompassing a 
respectively different second sub-array of pixels of 
said second image that contains a pixel located 

20 geometrically proximate to the location of a respective 
pixel of said first sub-array of pixels of said first 
images, the size of a first search window being larger 
than a first image pattern window; 

d) defining, within each first search 
25 window, a first comparison window sized in 

correspondence with said first image pattern window; 

e) for respectively different locations of 
a first comparison window within a respective first 
search window, comparing the contents of said first 

30 image pattern window with the contents of said first 
comparison window; 

f ) specifying which pixels among the image 
pixels of said second array represent locations on said 
contoured surface corresponding to the respective 
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pixels of said first sub-array of pixels of said first 
image/ in accordance with the comparison carried out in 
step (e) ; and 

g) determining the parallax between 
5 locations on said contoured surface as captured by said 
first image with corresponding locations on said 
contoured surface as captured by said second image in 
accordance with separations between respective pixels 
of said first sub-array and those image pixels of said 
10 second array specified in step (f ) . 

17. A method according to claim 16, wherein 
step (g) comprises the steps of: 

(gl) defining a boundary window about each of 
the pixels of said first sub-array, such that a 

15 respective boundary window contains a respective pixel 
of said first sub-array and a plurality of neighboring 
pixels and such that adjacent boundary windows border 
one another; and 

(g2) for each pixel of said first sub-array, 

20 for which a respective pixel of said second sub-array 
has been specified in step (f), determining the 
parallax between pixels contained in its associated 
boundary window and corresponding pixels of said second 
array in accordance with separations between respective 

25 pixels of said first sub-array and those pixels of said 
second array specified in step (f ) . 

18. A method according to claim 17, wherein 
step (g2) comprises the steps of: 

(g2-l) for each pixel belonging to a 
30 respective boundary window defined in step (gl) , 

locating a respective pixel of said second array in 
accordance with the separation between the pixel of 
said first sub-array belonging to said respective 
boundary window and a pixel of said second array 
35 specified in step (f); 



(g2-2) for each respective pixel of said 
second array located in step <g2-l) , defining a "second 
search window containing that pixel and a plurality of 
neighboring pixels; 

(g2-3) for each pixel of said respective 
boundary window for which a respective pixel of said 
second array has been located in step (g2-l) , defining 
a second image pattern window containing said each 
pixel and a plurality of neighboring pixels, such that 
the size of said second image pattern window is less 
than the' size of said second search window; 

(g2-4) defining, within said second search 
window, a second comparison window sized in 
correspondence with said second image pattern window; 

(g2-5) for respectively different locations 
of said second comparison window within said second 
search window, comparing the contents of said second 
comparison window with the contents of said second 
image pattern window; 

<g2-6) specifying which pixels among the 
image pixels of said second search window are 
associated with locations on said contoured surface 
represented by pixels of said boundary window, in 
accordance with the comparison carried out in step (g2- 
5) ; and 

(g2-7) determining the parallax between 
locations on said contoured surface as seen in said 
first image and corresponding locations on said 
contoured surface as seen in said second image in 
accordance with separations between respective pixels 
of said boundary window and image pixels of said second 
search window specified in step (g2-6) . 

19. A method according to claim 16, wherein 
step (c) comprises defining a plurality of first search 
windows among said second array of image pixels, each 
of said first search windows encompassing a 
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respectively different second sub-array of pixels of 
said second image that contains a pixel located 
geometrically proximate to the location of a respective 
pixel of said first sub-array of pixels of said first 
5 images , the size of a first search window being larger 
than a first image pattern window, 

step (d) comprises defining said first 
comparison window as a first correlation window sized 
in correspondence with said first image . pattern window, 

10 step (e) comprises, for respectively 

different locations of said first correlation window 
within a respective first search window, correlating 
the contents of said first correlation window with the 
contents of said first image pattern window, and 

15 wherein 

step (f) comprises specifying which pixel 
among the pixels of said first search window represents 
that location on said contoured surface corresponding 
to a respective pixel of said first sub-array of pixels 
20 of said first image, in accordance with the location of 
the said first correlation window within said first 
search window for which the value of the correlation 
produced by step (e) is largest. 

20. A method according to claim 19, wherein 
25 step (b) comprises measuring the contrast within said 

first image pattern window, and wherein steps (c) - (f ) 
are carried out with respect to a respective pixel of 
said first sub-array in response to the value of the 
measured contrast being greater than a prescribed 
30 contrast value, 

21. A method according to claim 19, wherein 
step (f) includes the steps of measuring the parallax 
between said specified pixels and associated pixels of 
said first sub-array, determining the average value of 
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said measured parallax for a plurality of first search 
windows, and discarding those specified pixels whose 
parallax differs from said average value of said 
measured parallax by a prescribed difference. 

22. A method according to claim 21, wherein 
said prescribed difference corresponds to the standard 
deviation. 

23. A method according to claim 19, wherein 
step (g) comprises the steps of: 

(gl) defining a boundary window about each of 
the pixels of said first sub-array, such that a 
respective boundary window contains a respective pixel 
of said first sub-array and a plurality of neighboring 
pixels and such that adjacent boundary windows border 
15 one another; and 

(g2) for each pixel of said first sub-array, 
for which a respective pixel of said second sub-array 
has been specified in step (f ) , determining the 
parallax between pixels contained in its associated 
20 boundary window and corresponding pixels of said second 
array in accordance with separations between respective 
pixels of said first sub-array and those pixels of said 
second array specified in step (f ) . 

24. a method according to claim 23, wherein 
25 step (g2) comprises the steps of: 

(g2-l) for each pixel belonging to a 
respective boundary window defined in step (gl), 
locating a respective pixel of said second array in 
accordance with the parallax between the pixel of said 
first sub-array belonging to said respective boundary 
window and a pixel of said second array specified in 
step (f); 

(g2-2) for each respective pixel of said 
second array located in step (g2-l) , defining a second 
search window containing that pixel and a plurality of 
neighboring pixels; 
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(g2-3) for each pixel of said respective 
boundary window for which a respective pixel of said 
second array has been located in step (g2-l) , defining 
a second image pattern window containing said each 
S pixel and a plurality of neighboring pixels, such that 
the size of said second image pattern window is less 
than the size of said second search window; 

(g2-4) defining, within said second search 
window, a second correlation window sized in 
10 correspondence with said second image pattern window; 

(g2-5) for respectively different locations 
of said second correlation window within said second 
search window, correlating the contents of said second 
correlation window with the contents of said second 
15 image pattern window; 

(g2-6) specifying which pixels among the 
image pixels of said second search window are 
associated with locations on said contoured surface 
represented by pixels of said boundary window, in 
accordance with the correlation carried out in step 
(g2-5); and 

(g2-7) determining the parallax between 
locations on said contoured surface as seen in said 
first image and corresponding locations on said 

25 contoured surface as seen in said second image in 

oc^ciaLAUHs oecween respective pixels 

of said boundary window and image pixels of said second 
search window specified in step (g2-6) . 

25. A method according to claim 24, wherein 

30 step (g2-3) comprises measuring the contrast within 
said second image pattern window, and wherein steps 
(g2-4) - (g2-7) are carried out with respect to pixels 
of a respective first boundary window in response to 
the value of the measured contrast within said second 

35 image pattern window being- greater than a predetermined 
contrast value. 
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26. A method according to claim 18, further 
including the step of: 

(h) for each of the pixels of said first sub- 
array for which corresponding pixels of said second 
5 array were not specified in step (f) each of such 
pixels of said first sub-array having an associated 
boundary window as defined in step <gl) , determining 
the parallax between pixels contained in said 
associated boundary window and corresponding pixels of 

10 said second array, in accordance with the parallax 

determined in step (g2-7) between locations on said 
contoured surface as seen in said first image and 
corresponding locations on said contoured surface as 
seen in said second image. 

15 27. a method according to claim 26, wherein 

step (h) comprises the steps of: 

(hi) for a respective pixel belonging to an 
associated boundary window, locating a corresponding 
pixel of said second array in accordance with the 

20 geometric relationship between said respective pixel 

and a neighboring pixel the parallax between which and 
a corresponding pixel of said second array has been 
determined; 

(h2) for each respective pixel of said second 
25 array located in step (hi) , defining a third search 
window containing that pixel and a plurality of 
neighboring pixels; 

(h3) for a respective pixel belonging to said 
associated boundary window, defining a third image 
30 pattern window containing said respective pixel and a 
plurality of neighboring pixels, such that the size of 
said image pattern window is less than the size of said 
third search window; 

(h4) defining, within said third search 
35 window, a third correlation window sized in 

correspondence with said third image pattern window; 
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(h5) for respectively different locations of 
said third correlation window within said third search 
window, correlating the contents of "said third 
correlation window with the contents of said third 
5 image pattern correlation window; 

(h6) specifying which pixels among the image 
pixels of said third search window are associated with 
locations on said contoured surface represented by 
pixels of said associated boundary window, in 
10 accordance with the correlation carried out in step 
(h5) ; and 

(h7) determining the parallax between 
locations on said contoured surface as seen in said 
first image and corresponding locations on said 

15 contoured surface as seen in said second image in 

accordance with separations between respective pixels 
of said associated boundary window and image pixels of 
said third search window specified in step (h6)\ 

28. A method according to claim 27, wherein 

20 step (hi) comprises, for a respective pixel belonging 
to an associated boundary window, locating a 
corresponding pixel of said second array in accordance 
with the geometric relationship between said respective 
pixel and a pixel of a neighboring boundary window. 

25 29. A method according to claim 27, further 

including the step of: 

(_) for each of remaining pixels of said 
first array, for which parallax with pixels of said 
second array has not been determined through step (h) , 

30 determining the parallax between such pixels and 

corresponding pixels of said second array in accordance 
with the parallax by 

(il) for a respective remaining pixel, 
locating a corresponding pixel of said second array in 

35 accordance with the geometric relationship between said 
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respective remaining pixel and an adjacent pixel the 
parallax between which and a corresponding pixel of 
said second array has been determined; 

(12) for each respective corresponding pixel 
5 of said second array located in step (il) , defining a 

fourth search window containing said respective 
corresponding pixel and a plurality of neighboring 
pixels; 

(13) for a respective remaining pixel, 

10 defining a fourth, image pattern window containing said 
respective remaining pixel and a plurality of 
neighboring pixels, such that the size of said fourth 
image pattern window is less than the size of said 
fourth search window; 

15 < i4 > defining, within said fourth search 

window, a fourth correlation window sized in 
correspondence with said fourth correlation window; 

(15) for respectively different locations of 
said fourth correlation window within said fourth 

20 search window, correlating the contents of said fourth 
correlation window with the contents of said fourth 
image pattern window; 

(16) specifying which pixels among the image 
pixels of said fourth search window are associated with 

25 locations on said contoured surface represented by 
remaining pixels of said first array, in accordance 
with the correlation carried out in step (i5) ; and 

(17) determining the parallax between 
locations on said contoured surface as seen in said 

30 first image and corresponding locations on said 

contoured surface as seen in said second image in 
accordance with separations between respective pixels 
of said first array and image pixels of said fourth 
search window specified in step (i6) . 
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30. A method according to claim 29, further 
comprising repeating steps (il) - (i7) using, 
correlation and search windows of progressively larger 
sizes. 

5 31. A method according to claim 29, further 

comprising repeating steps (il) - (i7) using a 
different traverse of said first and second pixel 
arrays . 

32. A method according to claim 16, further 
10 including the step of (e) determining parallax for sub- 
pixels of said image by interpolating the parallax 
values derived in step (d) . 

33. A method according to 32, wherein step 
(e) comprises determining a correlation value for an 

15 integral pixel step at a maximum value of parallax and 
at two points on either side of the maximum, and 
processing the correlation values to interpolate the 
position between integral pixel steps at which 'the 
local* maxima occurs. 

20 34. A method according to claim 16, further 

including the step of (e) determining elevations of 
locations on said contoured surface in accordance with 
the parallax determined in step (d) . 

35. For use with a digitized image 

25 processing system wherein first and second digitized 

images respectively representative of first and second 
views of a contoured suriace as seen from first and 
second respectively different look angles are 
generated, said first and second digitized images 

30 containing respective first and second arrays of image 
pixels, a method of determining parallax between 
locations on said contoured surface as seen in said 
first digitized image and corresponding locations on 
said contoured surface as seen in said second digitized 

35 image, comprising the steps of: 



WO 92/06444 



'PCT/US91/06908 



-48- 

a) defining, within said first array of 
image pixels, a sub-array of reference pixels, said 
sub-array of reference pixels being distributed over 
that portion of said contoured surface seen contained 

5 in both of said first and second digitized images; 

b) defining, about each respective 
reference pixel of said sub-array, a respective first 
image pattern window, such that a respective first 
image pattern window contains a respective reference 

10 pixel and a plurality of neighboring pixels; 

c) defining a plurality of first search 
windows distributed among said second array of pixels, 
each of said first search windows encompassing a 
respectively different plurality of pixels that 

15 contains a tentative tie point pixel located 

geometrically proximate to the location of a respective 
reference pixel of said sub-array, the size of a first 
search window being larger than a first image pattern 
window; 

20 d > defining, within each first search 

window, a first correlation window sized in 
correspondence with said first image pattern window; 

e) for respectively different locations of 
a first correlation window within a respective first 

25 search window, correlating the contents of said second 
correlation window with the contents of said first 
image pattern window; 

f) identifying which pixels, among the 
image pixels of said second array, represent locations 

30 on said contoured surface that correspond to the 

respective reference pixels sub-array, in accordance 
with correlations carried out in step (e) ; 

g) determining the parallax between 
respective identified tie point pixels and associated 

35 tentative tie point pixels of said second array 
identified in step (f); and 
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h) determining the parallax between 
locations on said contoured surface as seen in said 
first digitized image and corresponding locations on 
said contoured surface as seen in said second digitized 
5 image, in accordance with the parallax, determined in 
step (g) . 

36. A method according to claim 35, wherein 
step (h) comprises the steps of: 

(hi) defining a boundary window about each of 

10 said reference pixels, such that a respective boundary 
window contains a respective reference pixel and a 
plurality of neighboring pixels and such that adjacent 
boundary windows border one another; and 

(h2) determining the parallax between pixels 

15 contained in respective boundary windows and 
corresponding pixels of said second array, in 
accordance with the parallax determined in step (g) 
between respective tentative tie point pixels and 
identified tie point pixels identified in step (f ) . 

20 37. A method according to claim 36, wherein 

step (h2) comprises the steps of: 

(h2-l) for each pixel belonging to a 
respective boundary window defined in step (hi), 
locating a respective pixel of said second array in 

25 accordance with the parallax determined in step (g) ; 

(h2-2) for each respective pixel of said 
second array located in step (h2-l) , defining a second 
search window containing said each respective pixel and 
a plurality of neighboring pixels; 

30 (h2-3) for each pixel of a respective 

boundary window for which a respective pixel of said 
second array has been located in step (h2-l) , defining 
a second image pattern window containing said each 
pixel and a plurality of neighboring pixels, such that 

35 the size of said second image pattern window is less 
than the size of said second search window; 
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(h2-4) defining, within said second search 
window, a second correlation window sized in; 
correspondence with said second image pattern window; 

(h2-5) for respectively different locations 
5 of said second correlation window within said second 
search window, correlating the contents of said second 
correlation window with the contents of said second 
image pattern window; 

(h2-6) identifying which pixels among the 
10 image pixels of said second search window are 

associated with locations on said contoured surface 
represented by pixels of said boundary window, in 
accordance with the correlations carried out in step 
v"n2-5 ) ; and 

15 (h2-7) determining the parallax between 

locations on said contoured surface as seen in said 
first digitized image and corresponding locations on 
said contoured surface as seen in said second digitized 
image* in accordance with separations between respective 

20 pixels of said boundary window and image pixels of said 
second search window identified in step <h2-6) . 

38. A method according to claim 37, wherein 
step (b) comprises measuring the contrast within said 
first image pattern window, and wherein steps (c) - (g) 

25 are carried out with respect to a respective reference 
pixel in response to the value of the measured contrast 
being greater than a prescribed contrast value. 

39. A method according to claim 38, wherein 
step (f) includes the steps of measuring the parallax 

30 between said identified pixels and associated tentative 
tie point pixels, determining the average value of said 
measured parallax for a plurality of first search 
windows, and discarding those identified tie point 
pixels whose parallax differs from said average value 

35 of said measured parallax by a prescribed difference. 
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40, A method according to claim 39, wherein 
said prescribed difference corresponds to the standard 
deviation. 

41, A method according to claim 37, further 
5 including the step of: 

(i) for each of said reference pixels for 
which corresponding pixels of said second array were 
not identified in step (f ) , each of such reference 
pixels having an associated boundary window as defined 

10 in step (hi), determining the parallax between pixels 
contained in said associated boundary window and 
corresponding pixels of said second array in accordance 
with the parallax determined in step (h2-7) between 
locations on said contoured surface as seen in said 

15 first digitized image and corresponding locations on 

said contoured surface as seen in said second digitized 
image. 

42, A method according to claim 41, wherein 
step (i) comprises the steps of: 

20 (il) for a respective pixel belonging to an 

associated boundary window, locating a corresponding 
pixel of said second array in accordance with the 
geometric relationship between said respective pixel 
and a neighboring pixel the parallax between which and 

25 a corresponding pixel of said second array has been 
determined; 

(12) for each respective pixel of said second 
array located in step (il) , defining a third search 
window containing that pixel and a plurality of 

30 neighboring pixels; 

(13) for a respective pixel belonging to said 
associated boundary window, defining a third image 
pattern window containing said respective pixel and a 
plurality of neighboring pixels, such that the size of 

35 said third image pattern window is less than the size 
of said third search window; 
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(14) defining, within said third search 
window, a third correlation window sized in '- 
correspondence with said third image pattern window; 

(15) for respectively different locations of 
5 said third correlation window within said third search 

window, correlating the contents of said third 
correlation window with the contents of said third 
image pattern window; 

(16) specifying which pixels among the image 
10 pixels of said third search window are associated with 

locations on said contoured surface represented by 
pixels of said associated boundary window, in 
accordance with the correlation carried out in step 

(15) ; and 

15 < i7 ) determining the parallax between 

locations on said contoured surface as seen in said 
first digitized image and corresponding locations on 
said contoured surface as seen in said second digitized 
image in accordance with separations between respective 

20 pixels of said associated boundary window and image 
pixels of said third search window specified in step 

(16) . 

43. A method according to claim 42, wherein 
step (il) comprises, for a respective pixel belonging 

25 to an associated boundary window, locating a 

corresponding pixel of said second array in accordance 
with the geometric relationship between said respective 
pixel and a pixel of a neighboring boundary window. 

44. A method according to claim 42, further 
30 including the step of: 

(j) for each of remaining pixels of said 
first array, for which parallax with pixels of said 
second array has not been determined through step (i) , 
determining the parallax between such pixels and 
35 corresponding pixels of said second array in accordance 
with the parallax by 
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(jl) for a respective remaining pixel, 
locating a corresponding pixel of said second array in 
accordance with the geometric relationship between said 
respective remaining pixel and an adjacent pixel the 
5 parallax between which and a corresponding pixel of 
said second array has been determined; 

(j2) for each respective corresponding pixel 
of said second array located in step (jl), defining a 
fourth search window containing said respective 
10 corresponding pixel and a plurality of neighboring 
pixels; 

(j3) for a respective remaining pixel, 
defining a fourth image pattern window containing said 
respective remaining pixel and a plurality of 
15 neighboring pixels, such that the size of said fourth 
image pattern window is less than the size of said 
fourth search window; 

(j4) defining, within said fourth sea'rch 
window, a fourth correlation window sized in 
20 correspondence with said fourth image pattern window; 

(j5) for respectively different locations of 
said fourth correlation window within said fourth 
search window, correlating the contents of said fourth 
correlation window with the contents of said fourth 
25 image pattern window; 

(j6) specifying which pixels among the image 
pixels of said fourth search window are associated with 
locations on said contoured surface represented by 
remaining pixels of said first array, in accordance 
30 with the correlation carried out in step (j5) ; and 

(j7) determining the parallax between 
locations on said contoured surface as seen in said 
first image and corresponding locations on said 
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contoured surface as seen in said second digitized 
image in accordance with separations between, respective 
pixels of said first array and image pixels of said 
fourth search window specified in step (j6) . 
5 45. A method according to claim 44, further 

comprising repeating steps (jl) - (j7) using 
correlation and search windows of progressively larger 
sizes . 

46. A method according to claim 44, further 
10 comprising repeating steps (jl) - (j7) using a 

different traverse of said first and second pixel 
arrays . 

47. A method according to claim 35, further 
including the step of (e) determining parallax for sub- 
pixels of said image by interpolating the parallax 
values derived in step (d) . 

48. A method according to 47, wherein step 
(e) comprises determining a correlation value for an 
integral pixel step at a maximum value of parallax and 
at two points on either side of the maximum, and 
processing the correlation values to interpolate the 
position between integral pixel steps at which the 
local maxima occurs. 

49. A method according to claim 35, further 
25 including the step of (e) determining elevations of 

locations on said contoured surface in accordance with 
the parallax determined in step (d) . 
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